Unreal Video Production Pipeline
We hit a point at work where our two product offerings weren't covering certain edge cases - we needed content with the flexibility that Live Action 360° offered, but with the editability of CG. The infrastructure already supported video upload - so CG Launched a new medium: pre-rendered video.
Although we originally built the proof of concept in Maya using Vray, we inevitably opted to render in Unreal because of the metahuman and facial animation plugins, as well as the greatly reduced render times.
Its easy enough to make some concept art, but the trick to launching this at scale would be to reduce the time to create content so dramatically, that we were able to make content as fast as the Creative Directors could write it.
We were able to do just that - here's how we did it:
Characters
Metahuman has an almost infinite variety of characters available - with 70 (and growing) starter characters that can be mixed and matched, the variety is endless.
The only adjustments that need be made are additional clothing, which can be purchased or made through Marvelous Designer, and Hair, which can be designed in Maya's Xgen and transferred in.
Environments
Here we laid out a few options:
Photo spheres:
Photos can be cheaply sourced, either from existing libraries, AI generated, or through marketplaces like Envato Elements. We created objects to capture reflections and shadows, modelled any interactive props, and lit the stage. This gets saved as its own scene, so that environments can be swapped out while keeping the performance the same. In the example above, a photographer shot a location, and so we also captured photogrammetry of the table to get a nice lifelike feel.
2. Fully Modelled:
While quick to generate, the disadvantage of the above method is that you have little to no flexibility to edit the environment or change the camera angle. This next project utilized a modelled environment instead.
Body Animations
We explored lots of different motion capture options, but landed on Move.AI is being the most powerful and robust. Wonder Studio was a close second, but left the feet floating quite often and required a lot of animator time for cleanup.
Facial Animation
Unreal's new facial animation system made it the clear rendering engine winner. By equipping our actors with a motion capture helmet, we were able to get high quality facial animation with very little labor.
Here's my husband modelling a 3D printed one I made while testing the idea out:
And here's a playblast our stellar animators made using this tech stack: